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DOD Needs to Complete Developmental Testing 
Before Making Significant New Investments 


What GAO Found 

Cascading F-35 testing delays could cost the Department of Defense (DOD) 
over a billion dollars more than currently budgeted to complete development of 
the F-35 baseline program. Because of problems with the mission systems 
software, known as Block 3F, program officials optimistically estimate that the 
program will need an additional 5 months to complete developmental testing. 
According to best practices, credible estimates are rooted in historical data. The 
program’s projections are based on anticipated test point achievements and not 
historical data. GAO’s analysis—based on historical F-35 flight test data— 
indicates that developmental testing could take an additional 12 months (see 
table below). These delays could affect the start of the F-35’s initial operational 
test and evaluation, postpone the Navy’s initial operational capability, and delay 
the program’s full rate production decision, currently planned for April 2019. 


Assumptions Used to Determine Delays in F-35 Completion of Developmental Testing 

Assumptions 

F-35 program 
office 

GAO 

Monthly test point execution rate 

384 

220 

Test point additions 

42 percent 

63 percent 

Test point deletions 

13 percent 

10.8 percent 

Schedule growth estimates 

Developmental test completes 

October 2017 

May 2018 

Month slip 

5 months 

12 months 


Source: GAO analysis and presentation of Department of Defense data. | GAO-17-351 


Program officials estimate that a delay of 5 months will contribute to a total 
increase of $532 million to complete development. The longer delay estimated 
by GAO will likely contribute to an increase of more than $1.7 billion, 
approximately $1.3 billion of which will be needed in fiscal year 2018. 

Meanwhile, program officials project the program will need over $1.2 billion in 
fiscal year 2018 to start two efforts. First, DOD expects it will need over $600 
million for follow-on modernization (known as Block 4). F-35 program officials 
plan to release a request for Block 4 development proposals nearly 1 year before 
GAO estimates that Block 3F—the last block of software for the F-35 baseline 
program—developmental testing will be completed. DOD policy and GAO best 
practices state that requirements should be approved and a sound business 
case formed before requesting development proposals from contractors. Until 
Block 3F testing is complete, DOD will not have the knowledge it needs to 
present a sound business case for Block 4. Second, the program may ask 
Congress for more than $650 million in fiscal year 2018 to procure economic 
order quantities—bulk quantities. However, as of January 2017 the details of this 
plan were unclear because DOD’s 2018 budget was not final and negotiations 
with the contractors were ongoing. According to internal controls, agencies 
should communicate with Congress, otherwise it may not have the information it 
needs to make a fully informed budget decision for fiscal year 2018. Completing 
Block 3F development is essential for a sound business case and warrants 
funding priority over Block 4 and economic order quantities at this time. 
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U.S. GOVERNMENT ACCOUNTABILITY OFFICE 


April 24, 2017 
Congressional Committees 

With estimated acquisition costs of nearly $400 billion, the F-35 Lightning 
II—also known as the Joint Strike Fighter—is the Department of 
Defense’s (DOD) most costly and ambitious acquisition program. Through 
this program, DOD is developing and fielding a family of next generation 
strike fighter aircraft that integrate low observable (stealth) technologies 
with advanced sensors and computer networking capabilities for the 
United States Air Force, Marine Corps, and Navy as well as eight 
international partners.'' The F-35 family comprises the F-35A conventional 
takeoff and landing variant, the F-35B short takeoff and vertical landing 
variant, and the F-35C carrier-suitable variant. The program is 
approaching the end of development and will begin increasing production 
rates significantly over the next few years. According to current 
projections, the U.S. portion of the program will require acquisition 
funding of $54 billion over the next 5 years as it approaches full-rate 
production. The program is expected to require nearly $276 billion, or 
about $12 billion a year on average, through 2038 to complete 
development and procure a total of 2,457 aircraft. In addition, DOD 
estimates indicate that the F-35 fleet could cost over $1 trillion to operate 
and support over its lifetime. 

We have reported on F-35 issues for many years. Over time, we have 
reported significant cost, schedule, and performance problems and have 
made numerous recommendations for improvement. DOD has taken 
action to address many of our recommendations to varying degrees. See 
appendix I for a matrix of prior GAO reports, recommendations, and DOD 
actions. In addition, a list of related GAO products is included at the end 
of the report. 

The National Defense Authorization Act for Fiscal Year 2015 included a 
provision for GAO to review the F-35 acquisition program annually until 
the program reaches full-rate production. This is the second report under 
that provision. In this report, we assess (1) progress with remaining 


^The international partners are the United Kingdom, Italy, the Netherlands, Turkey, 
Canada, Australia, Denmark, and Norway. These nations contributed funds for system 
development and all but Canada have signed agreements to procure aircraft. In addition, 
Israel, Japan, and South Korea have signed on as foreign military sales customers. 
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program development and testing; (2) DOD’s proposed future plans for 
new F-35 acquisition investments; and (3) manufacturing progress, 
including supply chain performance. 

To conduct our work, we reviewed and analyzed cost performance 
reports, test data and results, program briefings, and internal DOD 
program analyses. We collected data on and discussed key aspects of F- 
35 development progress, including flight testing, with program 
management and contractor officials as well as DOD test officials and 
program test pilots. We analyzed historical test point execution rates and 
cost performance reports to determine the program’s average monthly 
progress and to identify the program’s average monthly costs over a 6- 
month span ending in September 2016. We used those averages along 
with the program’s remaining flight test points to calculate the amount of 
time and money the program will likely need to finish development. To 
assess the reliability of the test and cost data, we reviewed the supporting 
documentation and discussed the development of the data with DOD 
officials instrumental in producing them. To assess DOD’s proposed 
plans for future F-35 investments, we discussed initiatives with program 
and contractor officials. We analyzed DOD’s fiscal year 2017 budget 
request to identify costs and compared the acquisition plans for these 
initiatives to DOD policy and GAO acquisition best practices. We also 
collected and analyzed production and supply chain performance data 
from DOD, Lockheed Martin (the prime aircraft contractor), and Pratt & 
Whitney (the prime engine contractor) and assessed the reliability of the 
data by reviewing supporting documentation and interviewing 
knowledgeable agency officials. We determined that all of the data we 
used were sufficiently reliable for the purposes of this report. Appendix II 
contains a detailed description of our scope and methodology. 

We conducted this performance audit from June 2016 to April 2017 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 
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Background 


As we have previously reported, DOD began the F-35 acquisition 
program in October 2001 without adequate knowledge about the aircraft’s 
critical technologies or design.^ In addition, DOD’s acquisition strategy 
called for high levels of concurrency or overlap among development, 
testing, and production. In our prior work, we have identified the lack of 
adequate knowledge and high levels of concurrency as major drivers of 
the significant cost and schedule growth as well as performance shortfalls 
that the program has experienced since 2001.^ The program has been 
restructured three times since it began: first in December 2003, again in 
March 2007, and most recently in March 2012. The most recent 
restructuring was initiated in early 2010 when the program’s unit cost 
estimates exceeded critical thresholds established by statute—a condition 
known as a Nunn-McCurdy breach.'^ DOD subsequently certified to 
Congress in June 2010 that the program was essential to national 
security and needed to continue. DOD then began efforts to significantly 
restructure the program and establish a new acquisition program 
baseline. These restructuring efforts continued through 2011 and into 
2012, during which time the department increased the program’s cost 
estimates and extended its testing and delivery schedules. Since then 
costs have remained relatively stable. Table 1 shows the cost, quantity, 
and schedule changes from the initial program baseline and the relative 
stability since the new baseline was established. 


^GAO, Tactical Aircraft: Opportunity to Reduce Risks in the Joint Strike Fighter Program 
with Different Acquisition Strategy, GAO-05-271 (Washington, D.C.: Mar. 15, 2005). 

^GAO-05-271 , and GAO, Joint Strike Fighter: DOD Actions Needed to Further Enhance 
Restructuring and Address Affordability Risks, GAO-12-437 (Washington, D.C.: June 14, 
2012 ). 

^Section 2433 of title 10 of the United States Code, commonly referred to as Nunn- 
McCurdy, requires DOD to notify Congress whenever a major defense acquisition 
program’s unit cost experiences cost growth that exceeds certain thresholds. This is 
commonly referred to as a Nunn-McCurdy breach. Significant breaches occur when the 
program acquisition unit cost or procurement unit cost increases by at least 15 percent 
over the current baseline estimate or at least 30 percent over the original estimate. For 
critical breaches, when these unit costs increase at least 25 percent over the current 
baseline estimate or at least 50 percent over the original, DOD is required to take 
additional steps, including conducting an in-depth review of the program. Programs with 
critical breaches must be terminated unless the Secretary of Defense certifies to certain 
facts related to the programs and takes other actions, including restructuring the 
programs. 10 U.S.C. § 2433a. 
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Table 1: Changes in Reported F-35 Joint Strike Fighter Program Cost, Quantity, and Deliveries, 2001-2015 

October 

March 

December 

Percentage 

Percentage 

2001 initial 

2012 latest 

2015 

Change from 

Change from 

baseline 

baseline 

estimates 

2001 to 2012 

2012 to 2015 

Expected quantities (number of aircraft) 

Developmental 14 

quantities 

14 

14 

0 

0 

Procurement 2,852 

quantities 

2,443 

2,443 

-14 

0 

Total quantities 2,866 

2,457 

2,457 

-14 

0 

Cost estimates (then-year dollars in billions)^ 

Development 34.4 

55.2 

55.1 

60 

-0.18 

Procurement 196.6 

335.7 

319.1 

71 

-4.94 

Military construction 2.0 

4.8 

4.8 

140 

0 

Total program 233.0 

acquisition 

395.7 

379.0 

70 

-4.22 

Unit cost estimates (then-year dollars in millions)' 





Program acquisition 81 

161 

154.3 

99 

-4.16 

Average 69 

procurement 

137 

130.6 

99 

-4.67 

Estimated delivery and production dates 

Initial operational capability 2010-2012 

Undetermined^ 

2015-2018 

Undetermined 

5-6 years 

Full-rate production 2012 

2019 

2019 

7 years 

0 years 


Source: GAO analysis of Department of Defense (DOD) data. | GAO-17-351 


Note: The 2016 cost data were not available during the time of our review. 

^Annual projected cost estimates expressed in then-year dollars reflect inflation assumptions made by 
a program. 

‘’When the baseline was finalized, DOD had not yet identified new initial operational capability dates 
for the military services. 


As the program has been restructured, DOD has also reduced near-term 
aircraft procurement quantities. From 2001 and through 2007, DOD 
deferred the procurement of 931 aircraft into the future, and then again 
from 2007 and through 2012, DOD deferred another 450 aircraft. Figure 1 
shows how planned quantities in the near term steadily declined over 
time. 
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Figure 1: Changes in F-35 Joint Strike Fighter Near-Term Procurements 
Annual procurements of F-35 aircraft 


200 



The F-35 is DOD’s most costly acquisition program, and over the last 
several years we have reported on the affordability challenges facing the 
program. As we reported in April 2016, the estimated total acquisition cost 
for the F-35 program was $379 billion, and the program would require an 
average of $12 billion per year from 2016 through 2038. The program 
expects to reach peak production rates for U.S. aircraft in 2022, at which 
point DOD expects to spend more than $14 billion a year on average for a 
decade (see fig. 2). Given these significant acquisition costs, we found 
that DOD would likely face affordability challenges as the F-35 program 
competes with other large acquisition programs, including the B-21 
bomber, KC-46A tanker, and Ohio Class submarine replacement. In 
addition, in September 2014, we reported that DOD’s F-35 sustainment 
strategy may not be affordable.® 


®GAO, F-35 Sustainment: Need for Affordable Strategy, Greater Attention to Risks, and 
Improved Cost Estimates. GAO-14-778. (Washington, D.C.: Sept. 23, 2014). 
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Figure 2: F-35 Joint Strike Fighter Budgeted Deveiopment and Procurement Costs by Service 


Billions of dollars (then-year) 

16.2 



@ @ 

Aircraft purchased 

Navy procurement 
I I Navy development 

|_ j Air Force procurement 

Air Force development 

Source; GAO analysis of Department of Defense data. | GAO-17-351 


Through 2016, DOD had awarded contracts for production of 9 lots of F- 
35 aircraft, totaling 285 aircraft (217 aircraft for the U.S. and 68 aircraft for 
international partners or foreign military sales). At the time of this report, 
the contract for lot 10 had not been signed. 

In 2013, the Departments of the Navy and the Air Force issued a joint 
report to the congressional defense committees providing that the Marine 
Corps and Air Force would field initial operating capabilities in 2015 and 
2016, respectively, with aircraft that had limited warfighting capabilities. 
The Navy did not plan to field its initial operating capability until 2018, 
after the F-35’sfull warfighting capabilities had been developed and 


Page 6 


GAO-17-351 F-35 Joint Strike Fighter 



































































































tested.® These dates represented a delay of 5 to 6 years from the 
program’s initial baseline. As planned, the Marine Corps and Air Force 
declared initial operational capability (IOC) in July 2015 and August 2016, 
respectively. 


Continuing Problems 
with Flight Testing of 
Software Capabilities 
Will Delay the 
Completion of 
Development and 
Increase 

Development Costs 


DOD will need more time and money than expected to complete the 
remaining 10 percent of the F-35 development program. DOD has 
experienced delays in testing the software and systems that provide 
warfighting capabilities, known as mission systems, largely because the 
software has been delivered late to be tested and once delivered has not 
worked as expected. Program officials have had to regularly divert 
resources from developing and testing of more advanced software 
capabilities to address unanticipated problems with prior software 
versions. These problems have compounded overtime, and this past 
year was no exception. DOD began testing the final block of software— 
known as block 3F—later than expected, experienced unanticipated 
problems with the software’s performance, and thus did not complete all 
mission systems testing it had planned for 2016. As a result, the F-35 
program office has noted that more time and money will be needed to 
complete development. The amount of time and money could vary 
significantly depending on the program’s ability to complete 
developmental and operational testing. We estimate that developmental 
testing could be delayed as much as 12 months, thus delaying the start of 
initial operational testing, and total development costs could increase by 
nearly $1.7 billion.^ In addition, the Navy’s IOC and the program’s full-rate 
production decision could also be delayed. 


®Developmental testing is intended to provide feedback on the progress of a system’s 
design process and its combat capability as it advances toward initial production or 
deployment. Operational testing is intended to evaluate a system’s effectiveness and 
suitability under realistic combat conditions before full-rate production or deployment 
occurs. 

^ With regard to the costs for completing the development program, any estimates of 
additional needs made by the program office and GAO are based on the program office 
2018 preliminary budget projections. 
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Problems with Mission 
Systems Software 
Continue to Cause Delays 
in Flight Testing 


DOD continues to experience delays in F-35 mission systems testing. 
Although mission systems testing is about 80 percent complete, the 
complexity of developing and testing mission systems has been 
troublesome. For the F-35 program, DOD is developing and fielding 
mission systems capabilities in software blocks: (1) Block 1, (2) Block 2A, 
(3) Block 2B, (4) Block 3i, and (5) Block 3F. Each subsequent block builds 
on the capabilities of the preceding block. Over the last few years, 
program officials have had to divert resources—personnel and 
infrastructure—from developing and testing of more advanced software 
blocks to address unanticipated problems with prior software blocks. Over 
time, this practice has resulted in compounding delays in mission systems 
testing. Blocks 1 through 3i are now complete, and the program is 
currently focused on developing and testing Block 3F, the final software 
block in the current development program. Figure 3 illustrates the mission 
systems software blocks being developed for the program, the 
percentage of test points completed by block, and the build-up to full 
warfighting capability with Block 3F. 
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Figure 3: Development and Flight Test Status of F-35 Joint Strike Fighter Mission 
Systems Software Blocks as of December 2016 


Percentage test 
points complete 


Full warfighting capability 

Includes full avionics and weapons envelope 

Required for Navy 2018 Initial Operational Capability (IOC) 

Extension of Block 2B capabilities 

Includes adding Block 2B capabilities to new 
technology hardware, export compliance, and 
new helmet with improved display system 

Required for 2016 Air Force IOC 

Initial warfighting capability 

Includes basic close air support/interdictlon, 
and initial alr-to-air capability 

Required for 2015 Marine Corps iOC 


Training capability 

Includes basic navigation, mission planning, 
flight displays, voice communication, and 
threat jamming 


Source: GAO analysis of Department of Defense data. | GAO-17-351 


Block 3F 


Block 3i 


Block 2B 


Blocks 1 
and 2A 


100 % 


Program officials spent some of 2016 addressing problems with Block 3i 
mission systems unexpectedly shutting down and restarting—an issue 
known as instability—which delayed Block 3F testing. In early 2016, 
officials were developing and testing Block 3i concurrently with Block 3F. 
In order to ensure that the Block 3i instability was addressed in time for 
the Air Force’s planned IOC in August 2016, officials diverted resources 
from Block 3F.That decision delayed subsequent testing that had been 
planned for Block 3F. Further delays resulted from the discovery of 
instability and functionality problems with Block 3F. To mitigate some 
schedule delays, program officials implemented a new process to 
introduce software updates quicker than normal. Although the quick 
software releases helped to ensure that testing continued, the final 
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planned version of Block 3F, which was originally planned to be released 
to testing in February 2016, was not released until late November 2016, 
nearly a 10-month delay. As a result, program officials have identified the 
need for additional time to complete development. Program officials now 
project that developmental testing, which was expected to be completed 
in May, will conclude in October 2017, 5 months later than planned.® 

However, based on our analysis, the program’s projection is optimistic as 
it does not reflect historical F-35 test data. Program officials believe that 
going forward they may be able to devote more resources to mission 
systems testing, which could lead to higher test point completion rates 
than they have achieved in the past. According to GAO best practices, 
credible schedule estimates are rooted in historical data. As of November 
2016, program officials estimated that the program will need to complete 
as much as an average of 384 mission systems test points per month in 
order to finish flight testing by October 2017—a rate that the program has 
rarely achieved before.® Our analysis of historical test point data as of 
December 2016 indicates that the average test point execution rates are 
much lower, at 220 mission systems test points per month. In addition, 
historical averages suggest that test point growth—additions to the overall 
test points from discovery in flight testing—is much higher than program 
officials assume, while estimated deletions—test points that are 
considered no longer required—are lower than assumed. Using the 
historical F-35 averages, we project that developmental testing may not 
be completed until May 2018, a 12-month delay from the program’s 
current plan. Table 2 provides a comparison of the assumptions used to 
determine delays in developmental testing. 


^Program officials acknowledge a possible additional slip in developmental testing to 
February 2018 if flight test progress is slower than they expect. 

^Program officials also estimated that an average of 288 mission systems test points per 
month would allow the program to complete developmental testing in February 2018. In a 
memo sent to Congress in December 2016, the Under Secretary of Defense for 
Acquisition, Technology and Logistics stated that developmental testing could go as long 
as May 2018; however, the calculations provided by the program office do not reflect a 
schedule delay of this magnitude. 
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Table 2: Assumptions Used to Determine Delays in F-35 Completion of Developmental Testing 

Assumptions 

F-35 program office 

GAO 

Monthly mission systems test point execution rate 

384 

220 

Test point additions 

42 percent 

63 percent 

Test point deletions 

13 percent 

10.8 percent 

Schedule growth estimates 

Developmental test completes 

October 2017 

May 2018 

Month slip 

5 months 

12 months 


Source: GAO analysis and presentation of Department of Defense data. | GAO-17-351 


Our estimation of delays in completing developmental testing does not 
include the time it may take to address the significant number of existing 
deficiencies. The Marine Corps and Air Force declared IOC with limited 
capability and with several deficiencies. As of October 2016, the program 
had more than 1,200 open deficiencies, and senior program and test 
officials deemed 276 of those critical or of significant concern to the 
military services. Several of the critical deficiencies are related to the 
aircraft’s communications, data sharing, and target tracking capabilities. 
Although the final planned version of Block 3F software was released to 
flight testing in November 2016 and contained all 332 planned warfighting 
capabilities, not all of those capabilities worked as intended. In 
accordance with program plans, it was the first time some of the Block 3F 
capabilities had been tested. According to a recent report by the Director, 
Operational Test and Evaluation (DOT&E), fixes for less than half of the 
276 deficiencies were included in the final planned version of Block 3F 
software. Prime contractor officials stated that additional software 
releases will likely be required to address deficiencies identified during 
the testing of the final planned version of Block 3F software, but they do 
not yet know how many releases will ultimately be needed. 
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Delays in Developmental 
Testing Will Likely Affect 
Plans for Initial 
Operational Testing, Navy 
Operational Capability 
Dates, and Full-Rate 
Production 


Delays in developmental testing will likely drive delays in current plans to 
start F-35 initial operational test and evaluation. Program officials have 
noted that according to their calculations developmental testing will end in 
October 2017 and initial operational testing will begin in February 2018. 
However, DOT&E officials, who approve operational test plans, anticipate 
that the program will more likely start operational testing in late 2018 or 
early 2019, at the earliest. Figure 4 provides an illustration of the current 
program schedule and DOT&E’s projected delays. 


Figure 4: Director, Operationai Test and Evaiuation Office Anticipates Deiays to F-35 Joint Strike Fighter Deveiopmentai and 
Operationai Test and Evaiuation 


Fiscai year (FY) 
Quarter (Q) 



Q1 Q2 Q3 Q4 



Completion of 
Developmental 
Testing 

Program Current Schedule 
. Delayed to date 



Begin Initial 
Operational Test 
& Evaluation 



Completion of 
Initial Operational 
Test & Evaluation 




Q1 Q2 Q3 Q4 


Source: GAO presentation of Director, Operational Test and Evaluation data. | GAO-17-351 


Note: The current program schedule represents the latest information from the F-35 program office, 
while the delayed to dates are estimates from the Director, Operational Test and Evaluation Office. 


DOT&E’s estimate for the start of initial operational testing is based on 
the office’s projection that developmental testing will end in July 2018 and 
that retrofits needed to prepare the aircraft for operational testing will not 
be completed until late 2018 at the earliest. There are 23 aircraft—many 
of which are early production aircraft—that require a total of 155 retrofits 
before they will be ready to begin operational testing. As of January 2017, 
20 of those retrofits were not yet under contract, and program officials 
anticipated some retrofits would be completed in late 2018. 

To mitigate possible schedule delays, program officials are considering a 
phased start to operational testing. However, current program test plans 
require training and preparation activities before initial operational test 
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and evaluation begins. Those activities, as outlined in the test plan, are 
expected to take approximately 6 months. Changes to this approach 
would require approval from DOT&E. According to DOT&E officials, 
however, the program has not yet provided any detailed strategy for 
implementing a new approach or identified a time frame for revising the 
test plan. 

Significant delays in initial operational testing will likely affect two other 
upcoming program decisions: (1) the Navy’s decision to declare IOC and 
(2) DOD’s decision to begin full-rate production. In a 2015 report to the 
congressional defense committees, the Under Secretary of Defense for 
Acquisition, Technology and Logistics stated that the Navy’s IOC 
declaration is on track for February 2019 pending completion of initial 
operational test and evaluation.If initial operational testing does not 
begin until February 2019 as the DOT&E predicts, the Navy may need to 
consider postponing its IOC date. Likewise, DOD’s full-rate production 
decision, currently planned for April 2019, may have to be delayed. 
According to statute, a program may not proceed beyond low-rate initial 
production into full-rate production until initial operational test and 
evaluation is completed and DOT&E has submitted to the Secretary of 
Defense and the congressional defense committees a report that 
analyzes the results of operational testing. If testing does not begin until 
February 2019 and takes 1 year, as expected, DOD will not have the 
report in time to support a full-rate production decision by April 2019. 


Delays in Development 
Will Increase Program 
Costs and Add to 
Concurrency 


The current delays in F-35 developmental testing will also result in 
increased development costs. Based on the program office’s estimate of 
a 5-month delay in developmental testing, the F-35 program will need an 
additional $532 million to complete the development contract. According 
to GAO best practices, credible cost estimates are also rooted in 
historical data. Using historical contractor cost data from April 2016 to 
September 2016, we calculated the average monthly cost associated with 
the development contract. If developmental testing is delayed 12 months, 
as we estimate, and operational testing is not completed until 2020, as 
projected by DOT&E, then we estimate that the program could need more 


^°Department of Defense, Office of the Under Secretary of Defense for Acquisition, 
Technology and Logistics Joint Strike Fighter Software Deveiopment: Senate Report 113- 
76, Conference Report to Accompany S. 2410 (June 2015). 

^^10 U.S.C §2399. 
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than an additional $1.7 billion to complete the F-35 development contract. 
Similarly, the Cost Assessment and Program Evaluation office within the 
Office of the Secretary of Defense has estimated that the program will 
likely need more than $1.1 billion to complete the development contract. 

In these estimates, the majority of the additional funding would be needed 
in fiscal year 2018. Specifically, program officials believe that an 
additional $353.8 million may be needed in fiscal year 2018, while we 
estimate that they could need more than three times that amount— 
approximately $1.3 billion—as illustrated in figure 5.''^ 

The program plans to fund their estimated development program deficit 
through several means. For example, although the program office 2018 
preliminary budget projection reflected a reduction of $81 million in 
development funding over the next few years, as compared to DOD’s 
fiscal year 2017 budget request, program officials expect DOD to restore 
this reduction in its official fiscal year 2018 budget request. In addition, 
program officials plans to increase the budget request, as compared to 
their fiscal year 2017 budget request, for development funding in fiscal 
years 2018, 2019, and 2020 by $451 million and likewise reduce their 
budget request for procurement funding over those years. To make up for 
the reduction in requested procurement funding, the program plans to 
reprogram available procurement funds appropriated in prior fiscal 
years.Any additional funding beyond $451 million would likely have to 
come from some other source. Figure 5 compares DOD’s and our 
estimates for development funding needs from fiscal years 2018 through 
2021 . 


^^Fiscal year 2018 estimates are subject to change as DOD finalizes its budget request. 

1 

According to program officials, the majority of the reprogramming actions will be below 
the threshold that triggers prior approval of the congressional defense committees per 
DOD policy with the exception of $57.9 million in Marine Corps aircraft modification 
funding, which will be subject to an above threshold reprogramming that will require prior 
approval of the congressional defense committees per DOD policy. 
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Figure 5: F-35 Program Research, Development, Test and Evaluation Funding 
Estimates for Fiscal Years 2018 through 2021 
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As developmental testing is delayed and DOD procures more aircraft 
every year, concurrency costs—the costs of retrofitting delivered 
aircraft—increase. For example, from 2015 to 2016, the program 
experienced a $70 million increase in concurrency costs. This increase 
was partially driven by the identification of new technical issues found 
during flight testing that were not previously forecasted, including 
problems with the F-35C outer-wing structure and F-35B landing gear.'’'^ 
Problems such as these have to be fixed on aircraft that have already 
been procured. Thus far, DOD has procured 285 aircraft and has 
experienced a total of $1.77 billion in concurrency costs. Although testing 
is mostly complete, any additional delays will likely result in delays in the 
incorporation of known fixes, which would increase the number of aircraft 


^^Additional information on the F-35C outer wings and other technical risks can be found 
in app. III. 
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that will require retrofits and rework and further increase concurrency 
costs as more aircraft are procured. According to program officials, most 
of the retrofits going forward are likely to be software related and thus 
less costly. However, according to DOD’s current plan, 498 aircraft will be 
procured by the time initial operational testing is complete. If the 
completion of operational testing is delayed to 2020, as DOT&E predicts, 
the number of procured aircraft will increase to 584 as currently planned, 
making 86 additional aircraft subject to any required retrofits or rework. 


DOD’s Planned 
Investments in 
Modernization and 
Economic Order 
Purchase May Be 
Premature 


In fiscal year 2018, F-35 program officials expect to invest more than $1.2 
billion to start two efforts while simultaneously facing significant shortfalls 
in completing the F-35 baseline development program, as discussed 
above. Specifically, DOD and program officials project that in fiscal year 
2018 the program will need over $600 million to begin development of 
follow-on modernization of the F-35 and more than $650 million to 
procure economic order quantities (EOQ) of parts to achieve cost savings 
during procurement.^® Contracting for EOQ generally refers to the 
purchase of parts in larger more economically efficient quantities to 
minimize the cost of these items. DOD officials emphasized that the 
specific amount of funding needed for these investments could change as 
the department finalizes its fiscal year 2018 budget request. Regardless, 
these investments may be premature. Early Block 4 requirements, which 
represent new capabilities beyond the original requirements, may not be 
fully informed before DOD plans to solicit proposals from contractors for 
how they might meet the government’s requirements—a process known 
as request for proposal (RFP). According to DOD policy, the 
Development RFP Release Decision Point is the point at which a solid 
business case is formed for a new development program. Until Block 3F 
testing is complete, DOD will not have the knowledge it needs to develop 
and present an executable business case for Block 4, with reliable cost 
and funding estimates. 


Follow-on Modornization Due to evolving threats and changing warfighting environments, program 

May Be Too Early officials project that the program will need over $600 million in fiscal year 

2018 to award a contract to begin developing new F-35 capabilities, an 
effort referred to as follow-on modernization. However, the requirements 


The funding needs for follow-on modernization are based on DOD’s fiscal year 2017 
budget request while the projections for EOQ are based on discussions and comments 
from program officials. 
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for the first increment of that effort, known as Block 4, have not been 
finalized. Block 4 is expected to be developed and delivered in four 
phases—currently referred to as 4.1,4.2, 4.3, and 4.4. Program officials 
expect phases 4.1 and 4.3 to be primarily software updates, while 4.2 and 
4.4 consist of more significant hardware changes. The program has 
drafted a set of preliminary requirements for Block 4 that focused on the 
top-level capabilities needed in phases 4.1 and 4.2, but the requirements 
for the final two phases have not been fully defined. In addition, as of 
January 2017, these requirements had not been approved by the Joint 
Requirements Oversight Council.^® 

Delays in developmental testing of Block 3F are also likely to affect Block 
4 requirements. DOD policy states that requirements are to be approved 
before a program reaches the Development RFP Decision Point in the 
acquisition process.''^ GAO best practices emphasize the importance of 
matching requirements and resources in a business case before a 
development program begins.''® For DOD, the Development RFP Release 
Decision Point is the point at which plans for the program must be most 
carefully reviewed to ensure that all requirements have been approved, 
risks are understood and under control, the program plan is sound, and 
the program will be affordable and executable. Currently, F-35 program 
officials plan to release the RFP for Block 4.1 development in the third 
quarter of fiscal year 2017, nearly 1 year before we estimate Block 3F 
developmental testing will be completed. Program officials have stated 
that Block 3F is the foundation for Block 4, but continuing delays in Block 
3F testing make it difficult to fully understand Block 3F functionality and its 
effect on early Block 4 capabilities. If new deficiencies are identified 
during the remainder of Block 3F testing, the need for new technologies 
may arise, and DOD may need to review Block 4 requirements again 
before approving them. 


^®The Joint Requirements Oversight Council validates joint military requirements to ensure 
trade-offs among cost, schedule, and performance objectives are considered as part of 
DOD’s process for assessing and prioritizing requirements. 

^^DOD Instruction 5000.02, Operation of the Defense Acquisition System para. 5.d(5)(a)1. 
(Jan 7,2015) (incorp. change 1, eff. Jan. 26,2017). 

1 ft 

GAO, Best Practices: Better Matching of Needs and Resources Wiii Lead to Better 
Weapon System Outcomes, GAO-01-288 (Washington, D.C.: Mar. 8, 2001). 
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Costs and Benefits of 
Economic Order Quantity 
Purchases Are Not Clear 


In April 2016, we reported that the F-35 program office was considering 
what it referred to as a block buy contracting approach that we noted had 
some potential economic benefits but could limit congressional funding 
flexibility.''® The program office has since changed its strategy to consist 
of contracts for EOQ of 2 years’ worth of aircraft parts followed by a 
separate annual contract for procurement of lot 12 aircraft with annual 
options for lots 13 and 14 aircraft. Each of these options would be 
negotiated separately, similar to how DOD currently negotiates contracts. 


As of January 2017, details of the program office’s EOQ approach were 
still in flux. In 2015, the program office contracted with RAND Corporation 
to conduct a study of the potential cost savings associated with several 
EOQ approaches. According to the results of that study, in order for the 
government to get the greatest benefit, the aircraft and engine contractors 
would need to take on risk by investing in EOQ on behalf of the 
department in fiscal year 2017. Program officials envision that under this 
arrangement the contractors would be repaid by DOD at a later date. 
However, as of January 2017, contractors stated they were still 
negotiating the terms of this arrangement; therefore, the specific costs 
and benefits remained uncertain. Despite this uncertainty, the program 
office plans to seek congressional approval to make EOQ purchases and 
expects to need more than $650 million for that purpose in fiscal year 
2018. Program officials believe that this upfront investment would result in 
a significant savings over the next few years for the U.S. services. 
However, given the uncertainties around the level of contractor 
investment, it is not clear whether an investment of more than $650 
million, if that is the final amount DOD requests in fiscal year 2018, will be 
enough to yield significant savings. Regardless, with cost growth and 
schedule delays facing the F-35 baseline development program, it is 
unclear whether DOD can afford to fund this effort at this time. According 
to internal control standards, agencies should communicate with external 
stakeholders, such as Congress.^® With a potential investment of this 
size, particularly in an uncertain budget environment, it is important that 
program officials finalize the details of this approach before asking for 
congressional approval and provide Congress with a clear understanding 


^®GAO, F-35 Joint Strike Fighter: Continued Oversight Needed as Program Pians to Begin 
Deveiopment of New Capabiiities, GAO-16-390 (Washington, D.C.: Apr. 14, 2016). 

GAO, Standards for internat Controi in the Federai Government, GAO-14-704G 
(Washington, D.C.: Sept. 2014). 
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of the associated costs to ensure that funding decisions are fully 
informed. 


Aircraft Manufacturing 
and Reliability Are 
Improving in Some 
Areas 


The F-35 airframe and engine contractors continue to report improved 
manufacturing efficiency, and program data indicate that reliability and 
maintainability are improving in some areas. Over the last 5 years, the 
number of U.S. aircraft produced and delivered by Lockheed Martin has 
increased, and manufacturing efficiency and quality have improved over 
time. Similarly, manufacturing efficiency and quality metrics are improving 
for Pratt & Whitney. Although some engine aircraft reliability and 
maintainability metrics are not meeting program expectations, there has 
been progress in some areas, and there is still time for further 
improvements. 


Airframe and Engine 
Manufacturing Efficiency Is 
Improving, but Supply 
Chain Challenges Remain 


Overall the airframe manufacturer, Lockheed Martin, is improving 
efficiency and product quality. Over the last 5 years, the number of 
aircraft produced and delivered by Lockheed Martin has increased from 
29 aircraft in 2012 to 46 aircraft in 2016. Since 2011, a total of 200 
production aircraft have been delivered to DOD and international 
partners, 46 of which were delivered in 2016.^^ As of January 2017, 142 
aircraft were in production, worldwide.As more aircraft are delivered, 
the number of labor hours needed to manufacture each aircraft declines. 
Labor hours decreased from 2015 to 2016, indicating production maturity. 
In addition, instances of production line work done out of sequence 
remains relatively low, with the exception of an increase at the end of 
2016 due to technical issues, such as repairing coolant tube insulation 
(see app. III). Further, the number of quality defects and total hours spent 
on scrap, rework, and repair declined in 2016. 


Although data indicate that airframe manufacturing efficiency and quality 
continue to improve, supply chain challenges remain. Some suppliers are 
delivering late and non-conforming parts, resulting in production line 
inefficiencies and workarounds. For example, in 2016, Lockheed Martin 


Of the 200 aircraft, 173 have been delivered to the United States and 27 have been 
delivered to international partners, including 8 to the United Kingdom, 2 to the 
Netherlands, 4 to Norway, 7 to Italy, and 2 to Australia, and 4 to foreign military sales 
customers. 

^^Of the 142 aircraft currently in production, 82 are United States aircraft and 60 are for 
international partners or foreign military sales. 
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originally planned to deliver 53 aircraft, but quality issues with insulation 
on the coolant tubes in the fuel tanks resulted in the contractor delivering 
46 aircraft. According to Lockheed Martin officials, late deliveries of parts 
are largely due to late contract awards and supply base capacity. While 
supplier performance is generally improving, it is important for suppliers to 
be prepared for both production and sustainment support going forward. 
Inefficiencies, such as conducting production line work out of sequence, 
could be exacerbated if late delivery of parts continues as production 
more than doubles over the next 5 years. 

The engine manufacturer, Pratt & Whitney, is also improving efficiency. 

As of October 2016, Pratt & Whitney had delivered 279 engines. The 
labor hours required to assemble an F-35 engine decreased quickly and 
has remained relatively steady since around the 70th engine produced, 
and little additional efficiency is expected to be gained. Other Pratt & 
Whitney manufacturing metrics indicate that production efficiency and 
quality are improving. Scrap, rework, and repair costs were reduced from 
2.22 percent in 2015 to 1.8 percent in 2016. We previously reported that 
according to Pratt & Whitney officials, moving from a hollow blade design 
to a solid blade would reduce scrap and rework costs because it is easier 
to produce.However, Pratt & Whitney experienced unanticipated 
problems with cracking in the solid blade design. As a result, Pratt & 
Whitney is continuing to produce a hollow blade while it further 
investigates the difficulty and costs associated with a solid blade design. 
Pratt & Whitney’s supply chain continues to make some improvements. 

For example, critical parts are being delivered ahead of schedule, and 
some are already achieving 2017 rate requirements. To further ensure 
that suppliers are capable of handling full-rate production, Pratt & Whitney 
is pursuing the potential to have multiple suppliers for some engine parts, 
which officials believe will help increase manufacturing capacity within the 
supply chain. 

Aircraft Reliability and 
Maintainability Metrics 
Have Improved in Some 
Areas 

Although the program has made progress in improving system-level 
reliability and maintainability, some metrics continue to fall short of 
program expectations in several key areas. For example, as shown in 
figure 6, while metrics in most areas were overall trending in the right 
direction, the F-35 program office’s internal assessment indicated that as 
of August 2016 the F-35 fleet was falling short of reliability and 
maintainability expectations in 11 of 21 areas. 

2^GA0-1 6-390. 
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Figure 6: F-35 Program Office’s Assessment of the Status and Overaii Trend of F-35 
System-Levei Reiiabiiity and Maintainabiiity Metrics as of August 2016 
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Note: While mean flight hours between failure (design controlled) are associated with specific 
expected growth rates, no official expected growth rates exist for the other metrics. Instead, progress 
of these other metrics is tracked and compared against the required performance. 


Although many of the metrics remain below program expectations, some 
of the metrics have shown improvement over the last year, and time 
remains for continued improvements. For example, our analysis indicates 
that since 2015, the F-35A reliability has improved from 4.3 mean flight 
hours between failure attributable to design issues to 5.7 hours, nearly 
achieving the goal at system maturity of 6 hours. The F-35A mean flight 
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hours between maintenance event metric has also improved and is now 
meeting program expectations. As of August 2016, the F-35 fleet had only 
flown a cumulative total of 63,187 flight hours. The program has time for 
further improvement as the ultimate goals for these reliability and 
maintainability metrics are to be achieved by full system maturity, or 
200,000 cumulative flight hours across the fleet. The program also plans 
to improve these metrics through additional design changes. 

Engine reliability varied in 2016. In April 2016, we reported that Pratt & 
Whitney had implemented a number of design changes that resulted in 
significant improvements to one reliability metric: mean flight hours 
between failure attributable to design issues. At the time of our report, 
contractor data indicated the F-35A and F-35B engines were at about 55 
percent and 63 percent, respectively, of where the program expected 
them to be. According to contractor data as of September 2016, the 
program was unable to achieve a significant increase in reliability over the 
last year, which left the F-35A and F-35B engines further below 
expectations—at about 43 percent and 41 percent, respectively. Other 
reliability metrics such as engine’s impact on aircraft availability, engine 
maintenance man-hours, and the time between engine removals are 
meeting expectations. On average, from June 2016 through November 
2016, the engine affected only about 1.47 percent of the overall aircraft 
availability rates, and none of the top 30 drivers affecting aircraft 
availability were related to the engine. According to Pratt & Whitney 
officials, the F-35 engine requires fewer maintenance man-hours per flight 
hour than legacy aircraft, and engines for the F-35A and F-35B are 
currently performing better than required for the average number of flight 
hours between engine removals. Program and contractor officials 
continue to identify ways to further improve reliability through a number of 
design changes and expect reliability to continue to improve lot over lot. 


Conclusions 


As the F-35 program approaches the end of development, its schedule 
and cost estimates are optimistic. The program’s cost and schedule 
estimates to complete development are hundreds of millions of dollars 
below and several months under other independent estimates, including 
our own. If the program experiences schedule delays as we predict, it 
could require a total of nearly $1.5 billion in fiscal year 2018 alone. 
However, program officials project that the program will only need $576.2 
million in fiscal year 2018 to complete baseline development. At the same 
time, program officials expect that more than $1.2 billion could be needed 
to commit to Block 4 and EOQ in fiscal year 2018. DOD must prioritize 
funding for the baseline development program over the program office’s 
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desire for EOQ and Block 4. If baseline development is not prioritized and 
adequately funded, and costs increase as predicted by GAO and others, 
then the program will have less recourse for action and development 
could be further delayed. In addition, with baseline development still 
ongoing the program will not likely have the knowledge it needs to 
present a sound business case for soliciting contractor proposals for 
Block 4 development in fiscal year 2017. Although Block 4 and EOQ may 
be desirable, prioritizing funding for these efforts may not be essential at 
this time. Prioritizing funding for baseline development over these two 
efforts would ensure that the program has the time and money needed to 
properly finish development and thus lay a solid knowledge-based 
foundation for future efforts. 


Recommendations 
Executive Action 


1. Reassess the additional cost and time needed to complete 
developmental testing using historical program data. 

2. Delay the issuance of the Block 4 development request for proposals 
at least until developmental testing is complete and all associated 
capabilities have been verified to work as intended. 

3. Finalize the details of DOD and contractor investments associated 
with an EOQ purchase in fiscal year 2018, and submit a report to 
Congress with the fiscal year 2018 budget request that clearly 
identifies the details, including costs and benefits of the finalized EOQ 
approach. 


To ensure that DOD adequately prioritizes its resources to finish F-35 
baseline development and delivers all of the promised warfighting 
capabilities and that Congress is fully informed when making fiscal year 
2018 budget decisions, we are making the following three 
recommendations to the F-35 program office through the Secretary of 
Defense. 


Agency Comments 
and Our Evaluation 


DOD provided us with written comments on a draft of this report. DOD’s 
comments are reprinted in appendix IV and summarized below. DOD also 
provided technical comments, which were incorporated as appropriate. 

DOD did not concur with our recommendation to reassess the additional 
cost and time needed to complete developmental testing using historical 
program data. DOD stated that it will continue to assess the assumptions 
and decisions made, and communicate any necessary adjustments 
relative to both cost and time needed to complete developmental testing. 
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DOD also stated that it had considered historical data in its assessment 
and concluded that developmental testing could extend into February 
2018. While this possible slip is noted in our report, it is unclear to us the 
extent to which the data underpinning DOD’s assessment reflected the 
program’s historical averages. While the program’s analysis that we 
examined did reflect test point accomplishment rates that were more 
aligned with what the program achieved in 2016 (i.e. around 290 points 
per month) those rates were still higher than the historical average. Other 
key inputs to that analysis also differed significantly from the program’s 
historical averages. For example, program officials assumed only a 42 
percent test point growth rate when the program’s historical average test 
point growth was 63 percent, and in 2016 alone the test point growth rate 
was 115 percent. Several other DOD officials have identified possible 
delays beyond February 2018. In a memo sent to Congress in December 
2016, the Under Secretary of Defense for Acquisition, Technology and 
Logistics stated that developmental testing could go as long as May 2018, 
and DOT&E analysis also indicates that developmental testing may not 
conclude until mid-2018. We continue to believe that our recommendation 
is valid. 

DOD also did not concur with our recommendation to delay the issuance 
of the Block 4 development request for proposals until developmental 
testing is complete. According to DOD, delaying the request for proposals 
could unnecessarily delay delivery of needed capabilities to the 
warfighters. However, as program officials stated. Block 3F software 
establishes the foundation for Block 4. Therefore, continuing delays in 
Block 3F testing will likely make it difficult to fully understand Block 3F 
functionality and its effect on early Block 4 requirements. If new 
deficiencies are identified during the remainder of Block 3F testing, the 
need for new technologies may arise, and DOD may need to review Block 
4 requirements again before approving them which could lead to 
additional delays. Therefore, we continue to believe that our 
recommendation is valid. 

DOD stated that it partially concurred with our third recommendation to 
finalize the details of investments associated with an EOQ purchase in 
fiscal year 2018, and submit a report to Congress with the fiscal year 
2018 budget request that clearly identifies those details. However, in its 
response, the department outlined steps that address it. For example, 
DOD stated that it had finalized the details of DOD and contractor 
investments associated with an EOQ purchase and will brief Congress on 
the details, including costs and benefits of the finalized EOQ approach. 
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We are sending copies of this report to the appropriate congressional 
committees; the Secretary of Defense; and the Under Secretary of 
Defense for Acquisition, Technology and Logistics. In addition, the report 
is available at no charge on the GAO website at http://www.gao.gov. 

If you or your staff have any questions about this report, please contact 
me at (202) 512-4841 or sullivanm@gao.gov. Contact points for our 
Offices of Congressional Relations and Public Affairs may be found on 
the last page of this report. GAO staff who made key contributions to this 
report are listed in appendix V. 



Michael J. Sullivan 

Director, Acquisition and Sourcing Management 
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Appendix I: Prior GAO Report and DOD 
Actions 



Table 3: Prior GAO Reports on F-35 Joint Strike Fighter and Department of Defense (DOD) Responses and Subsequent 

Actions 

Year and 

GAO report 

Estimated 
development 
costs, development 
length, and aircraft 
unit cost 

Key program 
event 

Primary GAO 
conclusions/ 
recommendations 

DOD response 
and actions 

2001 

GAO-02-39 

• $34.4 billion 

• 10 years 

• $69 million 

Start of system 
development and 
demonstration 
approved. 

Critical technologies needed 
for key aircraft performance 
elements are not mature. 
Program should delay start of 
system development until 
critical technologies are 
mature to acceptable levels. 

DOD did not delay start 
of system development 
and demonstration 
stating technologies 
were at acceptable 
maturity levels and 
stated that it will 
manage risks in 
development. 

2005 

GAO-05-271 

• $44.8 billion 

• 12 years 

• $82 million 

The program 
undergoes re-plan to 
address higher-than- 
expected design 
weight, which added 
$7 billion and 18 
months to 
development 
schedule. 

We recommended that the 
program reduce risks and 
establish executable business 
case that is knowledge based 
with an evolutionary 
acquisition strategy. 

DOD partially 
concurred but did not 
adjust strategy, 
believing that its 
approach was balanced 
between cost, 
schedule, and technical 
risk. 

2006 

GAO-06-356 

• $45.7 billion 

• 12 years 

• $86 million 

Program sets in 
motion plan to enter 
production in 2007 
shortly after first flight 
of the non-production 
representative aircraft. 

The program was entering 
production with less than 1 
percent of testing complete. 

We recommended that the 
program delay investing in 
production until flight testing 
shows that the Joint Strike 
Fighter performs as expected. 

DOD partially 
concurred but did not 
delay start of 
production because it 
believed the risk level 
was appropriate. 

2007 

GAO-07-360 

• $44.5 billion 

• 12 years 

• $104 million 

Congress reduced 
funding for the first two 
low-rate production 
buys, thereby slowing 
the ramp-up of 
production. 

Progress was being made, 
but concerns remained about 
undue overlap in testing and 
production. We recommended 
limits to annual production 
quantities to 24 a year until 
flying quantities were 
demonstrated. 

DOD did not concur 
and stated that the 
program had an 
acceptable level of 
concurrency and an 
appropriate acquisition 
strategy. 
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Appendix I: Prior GAO Report and DOD 
Actions 


Year and 

GAO report 

Estimated 
development 
costs, development 
length, and aircraft 
unit cost 

Key program 
event 

Primary GAO 
conclusions/ 
recommendations 

DOD response 
and actions 

2008 

GAO-08-388 

• $44.2 billion 

• 12 years 

• $104 million 

DOD implemented a 
Mid-course Risk 
Reduction Plan to 
replenish management 
reserves from about 
$400 million to about 
$1 billion by reducing 
by reducing test 
resources. 

We found that the new plan 
increased risks and 
recommended that DOD 
revise it to address concerns 
about testing, management 
reserves, and manufacturing. 
We determined that the cost 
estimate was not reliable and 
recommended a new cost 
estimate and schedule risk 
assessment. 

DOD did not revise the 
risk plan or restore 
testing resources, 
stating that it will 
monitor the new plan 
and adjust it if 
necessary. Consistent 
with a report 
recommendation, a 
new cost estimate was 
prepared, but DOD did 
not conduct a risk and 
uncertainty analysis. 

2009 

GAO-09-303 

• $44.4 billion 

• 13 years 

• $104 million 

The program 
increased the cost 
estimate and added a 
year to development 
but accelerated the 
production ramp-up. 

An independent DOD 
cost estimate 
projected even higher 
costs and further 
delays. 

We concluded that moving 
forward with an accelerated 
procurement plan and use of 
cost reimbursement contracts 
was very risky. We 
recommended that the 
program report on the risks 
and mitigation strategy for this 
approach. 

DOD agreed to report 
its contracting strategy 
and plans to Congress 
and conduct a schedule 
risk analysis. The 
program reported 
completing the first 
schedule risk 
assessment with plans 
to update semiannually. 
The department 


announced a major 
program change 
reducing procurement 
and moving to fixed- 
price contracts. 


2010 

GAO-10-382 


$49.3 billion 
15 years 
$112 million 


The program was 
restructured to reflect 
findings from a recent 
independent cost team 
and independent 
manufacturing review 
team. As a result, 
development funds 
increased, test aircraft 
were added, the 
schedule was 
extended, and the 
early production rate 
decreased. 


Costs and schedule delays 
inhibited the program’s ability 
to meet needs on time. We 
recommended that the 
program complete a 
comprehensive cost estimate 
and assess warfighter and 
initial operational capability 
requirements. We suggested 
that Congress require DOD to 
tie annual procurement 
requests to demonstrated 
progress. 


DOD continued 
restructuring, 
increasing test 
resources, and 
lowering the production 
rate. Independent 
review teams evaluated 
aircraft and engine 
manufacturing 
processes. Cost 
increases later resulted 
in a Nunn-McCurdy 
breach. Military 
services are currently 
reviewing capability 
requirements, as we 
recommended. 
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Appendix I: Prior GAO Report and DOD 
Actions 


Year and 
GAO report 

2011 

GAO-11-325 


2012 

GAO-12-437 


Estimated 
development 
costs, development 
length, and aircraft 
unit cost 

• $51.8 billion 

• 16 years 

• $133 million 


• $55.2 billion 

• 18 years 

• $137 million 


Primary GAO 

Key program conclusions/ DOD response 

event recommendations and actions 


Restructuring 
continued with 
additional 
development cost 
increases, and 
schedule growth; 
further reduction in 
near-term procurement 
quantities; and a 
decreased rate for 
future production. The 
Secretary of Defense 
placed the short 
takeoff and vertical 
landing variant 
(STOVL) on a 2-year 
probation, decoupled 
STOVL from the other 
variants, and reduced 
STOVL production 
plans for fiscal years 
2011 to 2013. 

The program 
established a new 
acquisition program 
baseline and approved 
the continuation of 
system development, 
increasing costs for 
development and 
procurements and 
extending the period of 
planned procurements 
by 2 years. 


We concluded that the 
restructuring actions were 
positive and if implemented 
properly, should lead to more 
achievable and predictable 
outcomes. Concurrency of 
development, test, and 
production was substantial 
and provided risk to the 
program. We recommended 
that DOD maintain funding 
levels as budgeted; establish 
criteria for STOVL probation; 
and conduct an independent 
review of software 
development, integration, and 
test processes. 


Extensive restructuring placed 
the program on a more 
achievable course. Most of 
the program’s instability 
continued to be concurrency 
of development, test, and 
production. We recommended 
that the Cost Assessment 
Program Evaluation office 
conduct an analysis of the 
impact of lower annual 
funding levels, and that the 
program office conduct an 
assessment of the supply 
chain and transportation 
network. 


DOD concurred with all 
three of the 
recommendations. 

DOD lifted STOVL 
probation, citing 
improved performance. 
Subsequently, DOD 
further reduced 
procurement quantities, 
decreasing funding 
requirements through 
2016. The initial 
independent software 
assessment began, 
and ongoing reviews 
were planned to 
continue through 2012. 


DOD partially 
concurred with 
conducting an analysis 
of the impact of lower 
annual funding levels 
and concurred with 
assessing the supply 
chain and 

transportation network. 
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Year and 

GAO report 

Estimated 
development 
costs, development 
length, and aircraft 
unit cost 

Key program 
event 

Primary GAO 
conclusions/ 
recommendations 

DOD response 
and actions 

2013 

GAO-13-309 

• $55.2 billion 

• 18 years 

• $137 million 

The program 
continued to move 
forward following a 
new acquisition 
program baseline in 
2012. In doing so, the 
program incorporated 
positive and more 

The program was moving in 
the right direction but must 
fully validate design and 
operational performance and 
at the same time make the 
system affordable. We did not 
make recommendations to 
DOD in this report. 

DOD agreed with 

GAO’S observations. 


realistic restructuring 
actions taken since 
2010, including more 
time and funding for 
development and 
deferred procurement 
of more than 400 
aircraft to future years. 


2014 

GAO-14-322 


$55.2 billion 
18 years 
$135 million 


The services 
established initial 
operational capabilities 
dates in 2013. The 
Marine Corps and Air 
Force are planning to 
field initial operational 
capabilities in 2015 
and 2016, 

respectively, and the 
Navy plans to field its 
initial capability in 
2018. 


Delays in developmental flight 
testing of the F-35’s critical 
software may hinder delivery 
of the warfighting capabilities 
to the military services. We 
recommended that DOD 
conduct an assessment of the 
specific capabilities that can 
be delivered and those that 
will not likely be delivered to 
each of the services by their 
established initial operational 
capability dates. 


DOD concurred with 
our recommendation, 
and officials stated that 
they are in the process 
of conducting the 
assessment. 
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Estimated 
development 
costs, development 


Primary GAO 


Year and 

length, and aircraft 

Key program 

conclusions/ 

DOD response 

GAO report 

unit cost 

event 

recommendations 

and actions 


2014 

GAO-14-778 


2015 

GAO-15-364 


Not reported 


• $54.9 billion 

• 18 years 

• $136 million 


DOD was developing 
several plans and 
analyses that will 
make up its overall F- 
35 sustainment 
strategy, which was 
expected to be 
complete in fiscal year 
2019. 


Since the 2012 re¬ 
baselining, DOD has 
made changes to its F- 
35 procurement plans 
on an annual basis. 
The program also 
competed with other 
high-priority DOD 
programs for funding. 

In 2013 and 2014, 

DOD deferred a 
number of aircraft, 
extending the length of 
the program and 
increasing funding 
liability in the future. 


The annual F-35 operating 
and support costs were 
estimated to be considerably 
higher than the combined 
annual costs of several legacy 
aircraft. DOD had not fully 
addressed several issues that 
affect affordability and 
operational readiness. 
Operating and support cost 
estimates may not be fully 
reliable. GAO recommended 
that DOD develop better 
informed affordability 
constraints; address three 
risks that could affect 
sustainment, affordability, and 
operational readiness; and 
take steps to improve the 
reliability of its cost estimates. 

The consistent changes in F- 
35 procurement plans indicate 
that the analysis done to 
support the program’s 2012 
baseline did not accurately 
account for future technical 
risks or funding realities. We 
recommended that DOD 
conduct an affordability 
analysis of the current 
procurement plan that reflects 
various assumptions about 
technical progress and 
funding availability. 


DOD concurred with all 
but one 

recommendation and 
partially concurred with 
the recommendation to 
conduct uncertainty 
analysis on one of its 
cost estimates, stating 
that it already conducts 
a form of uncertainty 
analysis. 


DOD concurred with 
the recommendation 
and stated that it 
accomplishes an 
analysis of the 
program’s current 
procurement plans with 
various assumptions 
about technical 
progress and funding 
availability every year 
as it conducts reviews 
for the budget process. 
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Year and 

GAO report 

Estimated 
development 
costs, development 
length, and aircraft 
unit cost 

Key program 
event 

Primary GAO 
conclusions/ 
recommendations 

DOD response 
and actions 

2016 

GAO-16-390 

• $55.1 billion 

• 18 years 

• $130.6 million 

DOD planned to begin 
a block buy contracting 
approach that was 
anticipated to provide 
cost savings. In 
addition, DOD planned 
to manage the follow- 
on modernization 
program under the 
current F-35 program 
baseline and not as its 
own separate major 
defense acquisition 
program. 

The terms and conditions of 
the block buy and managing 
follow-on modernization under 
the current baseline could 
present oversight challenges 
for Congress. We 
recommended that the 
Secretary of Defense hold a 
Milestone B review and 
manage follow-on 
modernization as a separate 
major defense acquisition 
program. 

DOD did not concur 
with our 

recommendation. DOD 
viewed modernization 
as a continuation of the 
existing program and 
the existing oversight 
mechanisms, including 
regularly scheduled 
high-level acquisition 
reviews, will be used to 
manage the effort. 

2016 

GAO-16-439 

Not reported 

The Marine Corps 
declared initial 
operational capability 
in July 2015, while the 
Air Force and Navy 
plan to declare initial 
operational capability 
in 2016 and 2018, 
respectively. 

F-35 pilots and maintainers 
identified potential 
functionality risks to the 
Autonomic Logistics 

Information System (ALIS), 
and DOD lacks a plan to 
address these risks as key 
milestone dates approach. 

We recommended, among 
other things, that DOD 
develop a plan to address 

ALIS risks. 

DOD concurred with 
our recommendation to 
develop a plan to 
address ALIS risks, and 
work was under way 
that would form the 
foundation of the plan. 

Source: GAO | GAO-17-351 
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To assess the F-35 program’s remaining development and testing we 
interviewed officials from the program office and contractors—Lockheed 
Martin and Pratt & Whitney. We obtained and analyzed data on mission 
systems test point execution, both planned and accomplished from 2011 
through 2016 to calculate historical test point averages per month. We 
compared test progress against the total program requirements to 
determine the number of test points that were completed and remaining 
as of December 2016. We used the average test point rate based on the 
historical data to determine the number of months needed to complete 
the remaining test points. To identify the program’s average monthly 
costs, we analyzed contractor cost performance data from April 2016 
through September 2016 to identify average contract costs per month. 
Using a 12-month delay and the average contract costs per month, we 
calculated the costs to complete developmental testing. In order to 
determine costs to complete development, we first determined the 
percent change, year to year, in the program office’s development funding 
requirement estimate from 2018 to 2021. We then reduced our estimate 
using those percentages from 2018 to 2021. We discussed key aspects 
of F-35 development progress, including flight testing progress, with 
program management and contractor officials as well as DOD test 
officials and program test pilots. To assess the reliability of the test and 
cost data, we reviewed the supporting documentation and discussed the 
development of the data with DOD officials instrumental in producing 
them. In addition, we interviewed officials from the F-35 program office, 
Lockheed Martin, Pratt & Whitney, and the Director, Operational Test and 
Evaluation office to discuss development test plans, achievements, and 
test discoveries. 

To assess DOD’s proposed plans for future F-35 investments, we 
discussed cost and manufacturing efficiency initiatives, such as the 
economic order quantities approach, with contractor and program office 
officials to understand potential cost savings and plans. To assess the 
program’s follow-on modernization plans, we discussed the program’s 
plans with program office officials. We reviewed the fiscal year 2017 
budget request to identify costs associated with the effort. We also 
reviewed and analyzed best practices identified by GAO and reviewed 
relevant DOD policies and statutes. We compared the acquisition plans to 
these policies and practices. 

To assess ongoing manufacturing and supply chain performance, we 
obtained and analyzed data related to aircraft delivery rates and work 
performance data from January 2016 to December 2016. These data 
were compared to program objectives identified in these areas and used 
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to identify trends. We reviewed data and briefings provided by the 
program office, Lockheed Martin, Pratt & Whitney, and the Defense 
Contract Management Agency in order to identify issues in manufacturing 
processes. We discussed reasons for delivery delays and plans for 
improvement with Lockheed Martin and Pratt & Whitney. We collected 
and analyzed data related to aircraft quality through December 2016. We 
collected and analyzed supply chain performance data and discussed 
steps taken to improve quality and deliveries with Lockheed Martin and 
Pratt & Whitney. We also analyzed reliability and maintainability data and 
discussed these issues with program and contractor officials. 

We assessed the reliability of DOD and contractor data by reviewing 
existing information about the data and interviewing agency officials 
knowledgeable about the data. We determined that the data were 
sufficiently reliable for the purposes of this report. 

We conducted this performance audit from June 2016 to April 2017 in 
accordance with generally accepted government auditing standards. 
Those standards require that we plan and perform the audit to obtain 
sufficient, appropriate evidence to provide a reasonable basis for our 
findings and conclusions based on our audit objectives. We believe that 
the evidence obtained provides a reasonable basis for our findings and 
conclusions based on our audit objectives. 
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As developmental testing nears completion, the F-35 program continues 
to address technical risks. The program has incorporated design changes 
that appear to have mitigated several of the technical risks that we have 
highlighted in prior reports, including problems with the arresting hook 
system and bulkhead cracks on the F-35B. However, over the past year, 
the program continued to address risks with the Helmet Mounted Display, 
Autonomic Logistics Information System (ALIS), the ejection seat and 
engine seal that we have identified in the past. The program also 
identified new risks with the F-35C wing structure and catapult launches, 
and coolant tube insulation. The status of the Department of Defense’s 
(DOD) efforts to address these issues is as follows: 

Helmet Mounted Display: A new helmet intended to address shortfalls in 
night vision capability, among other things, was developed and delivered 
to the program in 2015. Developmental testing of the new helmet is 
mostly complete, and officials believe that issues such as latency and 
jitter have been addressed. Green glow, although improved, continues to 
add workload for the pilots when landing at sea. Officials believe that they 
have done as much as they can to fix the green glow problems with the 
hardware currently available. 

ALIS: ALIS continues to lack required capabilities; for instance, engine 
parts information is not included in the current version of ALIS, although it 
is expected to be completed in the spring of 2017. In 2016, officials began 
testing ALIS in an operational environment which has led to some 
improvements. However, capabilities, including the prognostics health 
management downlink, have been deferred to follow-on modernization. In 
2016, officials acknowledged compounding development delays and 
restructured the development schedule for ALIS. The new schedule 
shows that some capabilities that were planned in the earlier versions of 
ALIS will now be deferred to later versions. In April 2016, we reported that 
F-35 pilots and maintainors identified potential functionality risks to ALIS 
and that DOD lacked a plan to address these risks as key milestone 
dates approached, which could result in operational and schedule risks.'' 

Engine seal: Officials have identified a design change to address the 
technical problem that resulted in an engine fire in June 2014. This design 
change was validated and incorporated into production in 2015. Engine 


^GAO, F-35 Sustainment: DOD Needs a Plan to Address Risks Related to Its Central 
Logistics System. GAO-16-439 (Washington, D.C.: Apr. 14, 2016). 
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contractor officials identified 194 engines that needed to be retrofitted, 
and as of October 2016, 189 of those retrofits had been completed. The 
engine contractor, Pratt & Whitney, is paying for these retrofits. 

Ejection seat: In 2015, officials discovered that pilots who weigh less than 
136 pounds could possibly suffer neck injuries during ejection. Officials 
stated that the risk of injury is due to the over-rotation of the ejection seat 
in combination with the thrust from the parachute deployment during 
ejection. Officials noted that although the problem was discovered during 
testing of the new Helmet Mounted Display, the helmet’s weight was not 
the root cause. The program has explored a number of solutions to 
ensure pilot safety including installing a switch for light-weight pilots that 
would slow the release of the parachute deployment, installing a head 
support panel that would reduce head movement, and reducing the 
weight of the helmet. The final design completed qualification testing in 
2016 and is expected to be incorporated into production lot 10. The cost 
of these changes has not yet been determined. 

F-35C outer-wings: In 2016, officials identified structural issues on the F- 
35C outer-wing when carrying an AIM-9X missile. In order to resume the 
test program, officials identified a design change to include strengthening 
the wings’ material that was incorporated onto a test aircraft. Officials 
expect to incorporate retrofits to delivered aircraft by 2019 and will 
incorporate changes into production in lot 10. 

F-35C catapult launches: In 2016, officials identified issues with violent, 
uncomfortable, and distracting movement during catapult launches. 
Specifically, officials stated that the nose gear strut moves up and down 
as an aircraft accelerates to takeoff, which can cause neck and jaw 
soreness for the pilot because the helmet and oxygen mask are pushed 
back on the pilot’s face during take-off. This can be a safety risk as the 
helmet can hit the canopy, possibly resulting in damage, and flight critical 
symbology on the helmet can become difficult to read during and 
immediately after launch due to the rotation of the helmet on the pilot’s 
head. Officials evaluated several options for adjusting the nose gear to 
alleviate the issue, but determined that none of the options would 
significantly affect the forces felt by the pilot. Officials subsequently 
assembled a team to identify a root cause and a redesign. According to 
officials, adjustments to the catapult system load settings are being 
considered to address this issue, and a design change to the aircraft may 
not be required. But flight testing of the proposed changes is required to 
confirm this solution. 


Page 36 


GAO-17-351 F-35 Joint Strike Fighter 




Appendix III: Status of F-35 Technical Risks 


Insulation on coolant tubes: During maintenance on an aircraft in 2016, 
officials found that insulation around coolant tubes within the aircraft’s fuel 
system were cracking and contaminating the fuel lines. According to 
officials, the problem was a result of a supplier using the incorrect 
material for insulation. The faulty insulation was installed on 57 aircraft— 
including the entire Air Force initial operational capability fleet—which 
were prohibited from flight until the insulation was removed. Officials 
determined that the insulation would not need to be replaced as the 
aircraft meets specifications without it. Officials are considering removing 
the insulation from the tubes across the rest of the aircraft going forward. 
As of January 2017, all of the fielded aircraft have been repaired and 
returned to flight. 
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OFFICE OF THE UNDER SECRETARY OF DEFENSE 
3000 DEFENSE PENTAGON 
WASHINGTON, DC 20301-3000 


ACQUISITION, 

TECHNOLOGY. 

AND LOGISTICS 

APR 1 1 2017 

Mr. Michael J. Sullivan 
Director 

Acquisition and Sourcing Management 
U.S. Government Accountability Office 
441 G Street, NW 
Washington DC 20548 

Dear Mr. Sullivan: 

This letter serves as the Department of Defense (DoD) response to the Government 
Accountability Office (GAO) Draft Report. GAO-17-351, ‘T-35 JOINT STRIKE FIGHTER: 
DoD Needs to Complete Developmental Testing Before Making Significant New Investments,” 
dated March 3, 2017 (GAO Code 100905), The Department acknowledges receipt of the draft 
report. As more fully explained in the enclosure, the Department non-concurs with the first and 
second recommendations, and partially concurs with the third recommendation. 

The Department appreciates the opportunity to comment on the draft report. For further 
questions concerning this report, please contact Col Jason D. Voorheis. Tactical Warfare 
Systems, at 703-692-5604 or jason,d.voorheis.mil@mail.mil. 

Sincerely, 



James A. MacStravic 
Performing the Duties of the 
Under Secretary of Defense 
for Acquisition, Technology, 
and Logistics 

Enclosure: 

As stated 
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GAO DRAFT REPORT DATED APRIL 1, 2017 
GAO-17-351 (GAO CODE 100905) 

“F-35 JOINT STRIKE FIGHTER: DOD NEEDS TO COMPLETE DEVELOPMENTAL 
TESTING BEFORE MAKING SIGNIFICANT NEW INVESTMENTS” 

DEPARTMENT OF DEFENSE COMMENTS 
TO THE GAO RECOMMENDATION 


RECOMMENDATION 1: Reassess the additional cost and time needed to complete 
developmental testing using historical program data. 

DoD RESPONSE: Non-concur. DoD already completed a comprehensive assessment of the 
cost and time needed to complete developmental testing in the System Development and 
Demonstration (SDD) phase, and so far, the testing remains on track to complete in February 
2018. This assessment considered historical data and the recommendations from multiple DoD 
organizations. 

As the developmental testing winds further down during the remainder of 2017, DoD will 
continue to assess the assumptions and decisions made, and communicate any necessary 
adjustments relative to both cost and time needed to complete developmental testing. DoD is 
fully committed to completing the SDD phase using an event-based approach and delivering full 
Block 3F capability. 

RECOMMENDATION 2: Delay the issuance of the Block 4 development request for proposal 
at least until developmental testing is complete and all associated capabilities have been verified 
to work as intended. 

DoD RESPONSE: Non-concur. To account for contracting lead time, and to ensure a seamless 
transition from the SDD phase, release of the Request for Proposal (RFP) for Block 4 
development must take place prior to the end of developmental testing in the SDD phase. The 
DoD is presently assessing the overall Block 4 development schedule, in light of existing budget 
realities, to determine the appropriate date to release the RFP for Block 4 development. The 
expectation and target is to release the RFP prior to the end of 2017. 

Waiting until developmental testing is fully complete to release the RFP for Block 4 
development will introduce undue delay and negatively impact the Warfighter's ability to 
counter a wide spectrum of current and evolving Surface-to-Air Missile threats. Integrated Air 
Defense Systems, as well as current and emerging advanced fighter threats across 12 mission 
areas. These are identified in the Block 4 Capability Development Document, which was 
validated by the Joint Requirements Oversight Council on March 21, 2017. This delay will 
further prolong critical U.S. and Partner Warfighter capability gaps and significantly degrade the 
ability of the Warfighter to execute F-35 kill chains, decrease mission lethality, and decrease 

ENCLOSURE 

2 
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pilot survivability. This delay will also impact weapons certification for Partner-unique 
capability and cause them to miss critical, country-specific need dates. 


RECOMMENDATION 3: Finalize the details of DoD and contractor investments associated 
with an EOQ purchase in Fiscal Year (FY) 2018, and submit a report to Congress with the FY 
2018 budget request that clearly identifies the details, including costs and benefits of the 
finalized EOQ approach. 

DoD RESPONSE: Partially concur. DoD has finalized the details associated with its 
investment for an Economic Order Quantities (EOQ) purchase in FY 2018. DoD intends to 
submit a legislative proposal to Congress to authorize the EOQ purchase, and DoD will brief 
Congress on the details, including the costs and benefits of the finalized EOQ approach, as 
requested. 


ENCLOSURE 
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